Automatic Landmarking for Non-cooperative 3D Face Recognition

This thesis describes a new framework for 3D surface landmarking and evaluates its performance for feature localisation on human faces. This framework has two main parts that can be designed and optimised independently. The first one is a keypoint detection system that returns positions of interest for a given mesh surface by using a learnt dictionary of local shapes. The second one is a labelling system, using model fitting approaches that establish a one-to-one correspondence between the set of unlabelled input points and a learnt representation of the class of object to detect. Our keypoint detection system returns local maxima over score maps that are generated from an arbitrarily large set of local shape descriptors. The distributions of these descriptors (scalars or histograms) are learnt for known landmark positions on a training dataset in order to generate a model. The similarity between the input descriptor value for a given vertex and a model shape is used as a descriptor-related score. Our labelling system can make use of both hypergraph matching techniques and rigid registration techniques to reduce the ambiguity attached to unlabelled input keypoints for which a list of model landmark candidates have been seeded. The soft matching techniques use multi-attributed hyperedges to reduce ambiguity, while the registration techniques use scale-adapted rigid transformation computed from 3 or more points in order to obtain one-to-one correspondences. Our final system achieves better or comparable (depending on the metric) results than the state-of-the-art while being more generic. It does not require pre-processing such as cropping, spike removal and hole filling and is more robust to occlusion of salient local regions, such as those near the nose tip and inner eye corners. It is also fully pose invariant and can be used with kinds of objects other than faces, provided that labelled training data is available

3D face landmark labelling

Most 3D face processing systems require feature detection and localisation, for example to crop, register, analyse or recognise faces. The three features often used in the literature are the tip of the nose, and the two inner corner of the eyes. Failure to localise these landmarks can cause the system to fail and they become very difficult to detect under large pose variation or when occlusion is present. In this paper, we present a proof-of-concept for a face labelling system, capable of overcoming this problem, as a larger number of landmarks are employed. A set of points containing handplaced landmarks is used as input data. The aim here is to retrieve the landmark’s labels when some part of the face is missing. By using graph matching techniques to reduce the number of candidates, and translation and unit-quaternion clustering to determine a final correspondence, we evaluate the accuracy at which landmarks can be retrieved under changes in expression, orientation and in the presence of occlusions

Automatic keypoint detection on 3d faces using a dictionary of local shapes

Abstract—Keypoints on 3D surfaces are points that can be extracted repeatably over a wide range of 3D imaging conditions. They are used in many 3D shape processing applications; for example, to establish a set of initial correspondences across a pair of surfaces to be matched. Typically, keypoints are extracted using extremal values of a function over the 3D surface, such as the descriptor map for Gaussian curvature. That approach works well for salient points, such as the nosetip, but can not be used with other less pronounced local shapes. In this paper, we present an automatic method to detect keypoints on 3D faces, where these keypoints are locally similar to a set of previously learnt shapes, constituting a ‘local shape dictionary’. The local shapes are learnt at a set of 14 manuallyplaced landmark positions on the human face. Local shapes are characterised by a set of 10 shape descriptors computed over a range of scales. For each landmark, the proportion of face meshes that have an associated keypoint detection is used as a performance indicator. Repeatability of the extracted keypoints is measured across the FRGC v2 database. Keywords-3D face; keypoint detection; local 3D descriptors; I

Using the AGORASET dataset : assessing for the quality of crowd video analysis methods

International audienceIn this paper, we present a simulation-based crowd video dataset to be used for evaluation of low-level video crowd analysis methods, such as tracking or segmentation. Most of the time, an exact ground truth associated to real videos is difficult and time-consuming to produce, prone to errors, and these difficulties rise exponentially with the apparent density of the crowd in the image. We propose a synthetic crowd dataset to help researchers evaluate their methods against an objective and temporally dense synthetic ground truth. This dataset, named Agoraset, is presented in detail. The associated ground-truth and metrics are also described, together with a discussion on the use of this new kind of dataset in the field of pattern recognition. We believe this dataset to be the first bridge between simulation and pattern recognition in the field of dense crowd analysis. A discussion on the range of validity and limitations of the use of synthetic datasets in the contest of video processing is also proposed

Using the AGORASET dataset : assessing for the quality of crowd video analysis methods

International audienceIn this paper, we present a simulation-based crowd video dataset to be used for evaluation of low-level video crowd analysis methods, such as tracking or segmentation. Most of the time, an exact ground truth associated to real videos is difficult and time-consuming to produce, prone to errors, and these difficulties rise exponentially with the apparent density of the crowd in the image. We propose a synthetic crowd dataset to help researchers evaluate their methods against an objective and temporally dense synthetic ground truth. This dataset, named Agoraset, is presented in detail. The associated ground-truth and metrics are also described, together with a discussion on the use of this new kind of dataset in the field of pattern recognition. We believe this dataset to be the first bridge between simulation and pattern recognition in the field of dense crowd analysis. A discussion on the range of validity and limitations of the use of synthetic datasets in the contest of video processing is also proposed